@fry Thank you very much for taking out time to review. Here are my notes:

• L10N, I18N, A11Y: I think if we can pluck the text mapping file from the current PDE, it should be straightforward to set up inside the prototype.

• Package Manager: Yup, NPM comes by default but there's an alternative called yarn. It's not a replacement. It's just something you can use alongside. NPM should work just fine anyways.

• App Distribution: We can produce non-app store versions to start off with and come back to auto-updating and app distribution later once we get the prototype in a shippable state.

• Design: Sure, we can keep 3.x design. I think we'll have to write web styles from scratch. It should be pretty straightforward to do. If we have any old mocks about this design, please share. It's not needed but good to have for keeping the UI pixel perfect.

• Scope: I agree. CLI was the first thing that came to my mind but I knew we couldn't just pull off everything through it.

As you stated, there are two major slices of this project:

• feature parity/alignment with the PDE
• debug/autocomplete/etc code communication with the frontend

The feature parity issue is that while the PDE seems like a "simple" editor, there's a great deal of things that it can do, and that users expect it to do.

It would very helpful if you could clearly state what things we should cover for the first part other than the CLI integration with the code editor, UI design, web components and rest of the listed technical specifications.

Once we seal the scope for the first part, I'll make a detailed timeline and we'll see how much we can cover for the second part which is both trickier and complicated.

Again, thanks for the proposal. Hopefully we'll have a chance to work with you on the project this summer.

I feel electrified. This can't get more exciting. I would love to serve Processing. Thank you very much.

Thanks for this very thoughtful proposal. A couple quick notes, then the two big pieces are on the Design and Scope section. The quick things first:

• Base Framework: Agree that Electron is the way to go

• Code Editor: p5.js using CodeMirror (and your experience with that) sounds like the right thing

• UI/Frontend: React seems to be the thing

• Testing: sure, with an eye toward maintaining parity with the current PDE. Though it's also a lower priority than getting something working and may be out of scope for a summer-length project.

• L10N, I18N, A11Y: are all important, yes. You'll want to build on the work that's already been done for the PDE, as significant effort has gone into localizing the various messages in the PDE itself.

• App Distribution: we could do the app stores now, but do not, mostly because they require additional work and restrict features. They're also somewhat incompatible with the "sketchbook" model we have in place. Basically it'd be extra effort to support a more-closed system, and I'd still have to produce non-app store versions. Auto-updating would be nice too, but these are all laundry list features that would be outside what can happen in a GSoC project (relative to the other priorities).

• Language: sounds good… I go back and forth on TypeScript, and can be convinced either way.

• Package Manager: aren't you more-or-less stuck w/ NPM if you're using Electron anyway?

• Tooling: sure

And the longer points:

Design

We'll want to stick with the current 3.x design. The p5.js editor is a different project, and has a different audience and use case. We're looking for a direct port of the current state of the PDE that we can build on later. It's a mature application that's received a lot of work, has a lot of users, and a lot of resources and materials that describe the location and use of various features. Switching to the p5.js editor layout would be a step backwards: it's very early in its development and would remove too many things.

A general-purpose editor is also not a design goal—that's a distraction to getting something that works well for our use case. There are tons of editors out there, we don't need another generic one. And if anything, I think the p5.js editor and PDE should probably be diverging from one another—to be more specific to their use cases—rather than converging. (That last point is not a Foundation statement, it's me thinking out loud.)

Scope

The current CLI is very limited. It's really just the minimum to allow folks to run the app headless from their favorite editor. Wiring the CLI to a simple text editor that looks a bit like the current PDE as an electron app would be the first week or two of the project.

The CLI isn't really the way to go for anything further than that. Put another way, if this is just a frontend on the CLI, then why do it at all? We should just be sending people to more complete editor platforms and IDEs. The point of the PDE is how it ties the project together: making it easy for people to get started, to install and use libraries, to not have to think about the CLASSPATH, Java version, etc.

The real meat of making this project work is two things: first is feature parity/alignment with the PDE, and the second is how to make the debug/autocomplete/etc code work with the frontend.

The feature parity issue is that while the PDE seems like a "simple" editor, there's a great deal of things that it can do, and that users expect it to do. A lot of the complexity is hidden from users, but rebuilding it is an enormous task. I don't expect we'd have 100% parity with the PDE in a single GSoC project. (But of course I'd love to be wrong!)

For debug/autocomplete/etc, that's much trickier. You'd be working with Jakub and me to sort out how to best wire all that to a JS frontend, because it's both complicated and also needs some housecleaning along the way.

Again, thanks for the proposal. Hopefully we'll have a chance to work with you on the project this summer.

Development Environment: JavaScript Prototype

Dhruvdutt Jadhav
GitHub: dhruvdutt | Website: dhruvdutt.github.io | Email: dhruvdutt.jadhav@gmail.com

---

Introduction / Abstract

The Processing Development Environment (PDE) has more than 250k unique users and is currently built in Java using custom components that make use of Java's “Swing” library. This proposal aims a prototype replacement for the PDE itself with a JavaScript-based solution.

---

Background

The Processing IDE has 2 major components: - Code Editor: 'frontend' - Compiler/runner: 'backend' that converts processing code to pure Java code, compiles and runs the program.

The current PDE user interface is built in Swing and the code editor is a standard text view. When we click "run", the PDE converts Processing code to pure Java source code and then there's a 'runner' component that does the job of actually running the Java program. Everything except this part will need to be basically written from scratch.

---

Project Description

The prototype we're planning would be a native desktop app built using web technologies like JavaScript. It would have a Processing code editor and will allow the user to compile and run the code through the Java compiler or runner.

---

Approach

For the prototype, we can imagine a basic version of VSCode or Atom like editor which lets you code in Processing language.

We will use a cross-platform framework, integrate a code-editor and rewrite the web components. More details are given below in technical specifications.

For the parsing the code, we'll need to pass it through the runner/compiler which resides inside the monolith PDE repo. Processing does have a CLI (command line tool) that basically takes a sketch folder and runs it in a "headless" version of Processing (without launching the full PDE). We can leverage or tweak that to our use for the prototype.

The UI editor and components can stay JavaScript based and will communicate with the Processing CLI to run/pause/stop the sketch. We can use Node.js Child Process to accomplish this and use the stdout from CLI through the child process (inside Electron) and then pop it.

An alternative to using CLI is to actually translate the sketch code to pure Java, compile and run it as a standard Java program. Processing's core libraries are standard java libraries, so it's possible to run it as a pure Java program too. But it will be significantly more work than using the CLI directly. We would need to use something like node-java to connect with existing Java APIs and form a bridge but it might take a while in itself to figure out and stitch all the internal libraries together before doing any communication.

I think right now, we can focus on the exposing more and more features via CLI as per the requirements of the prototype. This way, we can keep developing the current PDE and the improving CLI together, and at a future stage when things are stable enough we can entirely ditch the Swing components.

We can keep the CLI as a bridge to only do the essential compilation and parsing of the Processing code and implement the rest of the features within the new prototype gradually.

Apart from this, the PDE also supports error reporting, code completion (intellisense), refactoring features too. These features currently use other 3rd party Java libraries. So porting these features over might be a bit tricky initially. These could be implemented using libraries like Tern.js but we can keep this at the back of our minds and cover this once we get the initial prototype ready.

Building the complete PDE would definitely take lots of effort. I'm seeing this proposal as building a strong bedrock foundation on which we can scale and iterate incrementally.

One of the goals to build this new prototype is moving away from Java's Swing components code and embrace JavaScript for frontend and UI components. JavaScript is best known for it's terse declarative, flexible and dynamic behaviour which makes it easy to pick and faster to iterate. It will also attract a more active open-source community and will lower the bar of entry for contributions.

---

Technical Specifications:

Base Framework

A base framework that allows building native desktop applications using web technologies like JavaScript, HTML, CSS.

We'll use Electron as our base framework as it's one of the most popular projects used for building native applications and is backed by GitHub. Tools like VSCode, Atom are built on Electron which has similar use case as us.

Code Editor

For taking Processing code as input from user, we'll need a code editor that runs on web technologies (or an in-browser code editor).

There are again a bunch of choices here but I've picked CodeMirror which is one of the oldest and most actively maintained code editors. It has support for tons of languages out-of-the-box, has an active community forum and has way more users and downloads (npm downloads) even after combining all of its alternatives. Live use cases similar to us are Adobe Brackets and Chrome DevTools (in case we plan to build some console utilities at a later stage). Also, our very own Processing family's p5.js web editor uses CodeMirror as well.

CodeMirror has out-of-the-box support for Java which we can start off with and it also features something called "mode system" that allowing supporting custom language syntax highlighting and markup which we can use for our Processing code inside the editor. There's also a 3rd party library called CodeMirror Grammar which transforms a JSON grammar object into a syntax-highlight parser for CodeMirror. We can also look into it.

In terms of code editors, there are alternatives like Monaco which is built by the VSCode team at Microsoft. It's the go-to editor if the target users would be working primarily for JavaScript inside the editor. Monaco doesn't support many languages out-of-the-box but they have very good docs about writing custom syntax and languages.

UI Components

"The current custom components make use of Java's “Swing” library, which is even more outdated now than when it was first released".

We'll need to rewrite the UI components from scratch which should be fairly straightforward and can be concretely planned. We'll try to use native things as much as we can, like the menu bar which would be native and look platform specific (similar to how VSCode, Atom does). The alternative is we build a custom menu bar with custom styles that would be inline and sticky in the header at the top of the IDE. (similar to Google Docs/Sheets menu bar)

Frontend Library

We'll also need a frontend library to connect and render all the UI interfaces and for managing application state. I've chosen React.js as it provides a simple yet flexible API for building user interfaces, follows components based architecture pattern, wildly popular, actively developed, backed by Facebook and has one of the biggest JavaScript communities. Redux: For application state management, Reselect for memoizing selectors if needed. React and Redux is also used in the p5.js web editor.

Design

We can take inspiration from the p5.js web editor design looks minimalistic and beginner-friendly. We can port these styles directly to keep things consistent and tweak a few things like Processing's dark blue colour theme and logo. Also, it is recommended to create a standalone design kit components library in future that would contain common components styled along with a way to inherit/extends them. This can be consumed by both p5.js web editor and the new PDE or any future web-based project and can serve as a design spec for all Processing web projects.

Testing

Testing is the roof of any app. As the app grows, testing will help us scale and iterate faster. Lacks of tests has been one of the issues with the current PDE. It's hard to maintain them with limited resources. For the new prototype, most of the internal methods and APIs will keep on changing/refactoring. It doesn't make sense to follow TDD from Day 0. We would try to write modular code in a way which facilitates easy testing at a later stage once the internal APIs and methods are in a stable state.

Unit testing would be good to start off with. There are testing frameworks specialized for each type of tests but Jest works for most cases. It is backed by Facebook and is wildly popular in React - JavaScript ecosystem. This is something which can be revisited at a later point in time.

Localization, Internationalization and Accessibility

Since the PDE would be targetting a wide audience, l10n, i18n and a11y can't be neglected and are one of the most crucial things to support. This is also something in backlog at the moment till we decide the priority of these in context of the current prototype.

App Distribution

Currently, the PDE is only distributed via the downloads page and GitHub releases page. The PDE prompts you with an update message if a new version is available.

With the help of Electron, we can distribute it to Mac App Store, Windows Store and also do the update automatically in the background (similar to how VSCode, Atom updates itself). There would be platform specific executables/msi/zips/tgz for sure along with this.

Although app distribution as a whole can be addressed at a much later stage in the project, we should keep in mind the possibilities which could be unlocked.

Language

For the programming language, we'll be using JavaScript/ECMAScript ES6/7 with Babel (transpiler/JS compiler) which will allow us to write modular code using latest syntax, semantics.

I'm also open to and have experience using statically typed languages like TypeScript which might help to get Java developers onboard but IMO we should stick with JavaScript.

Package Manager

For managing and installing all the packages and dependencies inside our project, we'll use npm which comes bundled with Node.js We can also use Yarn alongside npm.

Tooling

We'll also using some additional tools along with all above libraries:

• For module bundling, we'll go with webpack. It will help us using ECMAScript modules, tree shaking (dead-code elimination), uglification, minification, code-splitting, etc.

• Linting: ESLint and Prettier to maintain code consistency and formatting code.

• Continous Integration: Setup Travis CI with npm scripts for automatic testing, linting, etc.

• Other Internal tools: JSDocs, pre-commit hooks with Husky, lint-staged, etc.

---

Summary

• The whole technical stack comprises of most popular and battled-tested tools having similar live use-cases as ours. Personally, I'm acquainted and have worked with all of the technologies listed.

• The technologies are very similar to the current p5.js web editor. This will help improve developers and contributors experience switching between projects.

• We can expect to see a good number of outside JavaScript contributions from one of the biggest open-source communities.

• The major objectives we've covered/unlocked are:

  • Allow more people to contribute to PDE's further development.
  • Provide better support for building the UI by using JavaScript.
  • Have more visually consistent cross-platform results.
  • Potentially open up other ways to distribute the PDE.

---

Scope

For the initial prototype, we can imagine a basic version of the current PDE with a code editor and an option to run the sketch. The technical plan proposed forms a good foundation for a future scalable product. Most of the UI components would be covered in the prototype itself and once we form the communication bridge with the CLI properly, we can incrementally port and migrate the rest of the features.

Development Process (Timeline)

The timeline will be planned once this initial proposal is approved and we finalize the things to cover in the initial prototype.

Discussion

• What is the current state of the Processing CLI in terms of stability and how regularly is it maintained?

• Are there any features that currently aren't supported/exposed by the CLI which we want to accomplish in the prototype version? Currently, the --help command shows these supported options.

• Finalize the scope and things to cover in the prototype.

---

About me whoami

I'm Dhruvdutt Jadhav, a software engineer and a student from DAIICT, India. I'm currently in my final year of masters and have also completed React Nanodegree from Udacity.

I'm one of the co-authors and maintainers of ReactiveSearch - a React based web and native UI components library for Elasticsearch. I'm also one of the members and collaborators of webpack team and have done noteworthy contributions to webpack-cli. I've also done other minor contributions to yarn, yarn-website, create-react-native-app, react-router and many other open-source libraries. You can catch all my contributions on GitHub.

I've also contributed to Processing's p5.js-web-editor in submitting some patches and upgrading major packages.

Apart from this, I've built many web-based production apps and have primarily worked with JavaScript and React in my most closed source projects. You can learn more about them in my resume.

I'm very proficient working with JavaScript and web technologies in general. I've also worked with Java at an intermediate level and have built an open-source library to manage multiple filesystems as a part of my college project. Know more about it here.

I learnt about Processing a while ago and would love to be a part of its family for GSoC and beyond.

My details:

• GitHub: dhruvdutt
• Website: dhruvdutt.js.org
• Email: dhruvdutt.jadhav@gmail.com
• Timezone: UTC +5.30 (India). I'm a nocturnal person but open to work under any timezone.

---

Note: Big shout-out to Manindra Moharana for helping me throughout with the approach.

Thank you very much everyone for taking out time to read my proposal. I really appreciate your feedback and guidance about it.

If you don't want to pay Apple developer fees then you can't make an App store distributed app.

Instead, you could develop it as a web app using p5.js, then open it on an iPad using a web browser. Don't need a license fee to load a website.

Howdy, I'm new to processing and new to Androi so I suspect i have a conceptual block here. I've written a very simple routine that is trying to load a background image. Code Follows. It runs fine in Java but when I try to run it on the Android Emulator I'm getting a

java.io.FileNotFoundException: /data/user/0/processing.test.wrapmate/files/bg.jpg (No such file or directory)

I'm assuming this is because my image file is sitting on my computer and not on my Device (either a real one or the Emulator).

So a) during development, what's the way to get any image files I need to access to the right place on the Android so that the program can access them?

b) If I eventually want to "deploy" an app to my friend's devices (like through the App Store?) how do I ensure that any image files come along with the application?

Thanks!

Dave

edit... trying to learn to format code in the forum. Sorry!

void setup()
{
  size(displayWidth, displayHeight, P3D);
}

void draw()
{
  //noStroke();
  PImage back; 
  back = loadImage("bg.jpg");
  back.resize (displayWidth, displayHeight); 
  background(back);

  float box_width = 60;

  if (mousePressed) {
    box_width = mouseX;
  }

  lights();
  translate(width/2, height/2, 0);
  rotateX(-PI/6);
  rotateY(PI/2 + 210/float(height) * PI);
  box(box_width, displayHeight/6, displayWidth/3);
}

The link is for android. I've posted twice in App store and PC and mac issues. No response. Mails to them have also not elicited any. Why I'm posting in Processing is because the response from Amazon team is that the exported app (exe) is not launching whereas it works fine in my PC.

Depends on which App Store -- Here are some recent examples of Android games:

• Kōan: An Art Game https://forum.processing.org/two/discussion/21809/koan-an-art-game-made-with-android-studio-processing
• Zoe Line https://forum.processing.org/two/discussion/21329/zoe-line-processing-android-game-on-the-google-play-you-can-download-it/p1
• polyrhythmic metronome https://forum.processing.org/two/discussion/22142/polyrhythmic-metronome-for-android

In general, search the "Share Your Work" area or search for venue names like "Google Play" or "App Store" to find examples listed on the forums.

• https://forum.processing.org/two/categories/share-your-work
• https://forum.processing.org/two/search?Search="Google+Play"
• https://forum.processing.org/two/search?Search="App+Store"

Here are some posts about the process -- it is not as simple as "export", you need to develop the whole app in a different way. Android is doable, Processing to iOS is very very difficult.

• https://forum.processing.org/two/discussion/18271/is-it-possible-to-upload-program-to-google-app-store/p1
• https://forum.processing.org/two/discussion/19274/why-is-there-no-ios-section-for-p5
• https://forum.processing.org/two/discussion/19471/processing-for-ios
• https://forum.processing.org/two/discussion/6887/port-processing-for-android-app-to-ios

Anyone ever been able to export an app and get it on the App Store by chance?

You can write your app in processing. I believe you can generate the app directly from processing and submitted to the App Store (I haven't done it myself). Another way is to write your app in Eclipse or AS and use the processing libraries there. It is all the same as you write your app in Java. You just need to stick to an IDE to start. One thing I could add is that if you write your app in processing, you can export it into Eclipse.

What you need to do is to learn about the life cycle of the apps and become familiar with the Android API and any other tool required for your app. So yes, you can work in Processing and focus in the java part.

Kf

Since processing is just Java, you can check in the App Store for any app that can read java code. Here is my google search:

https://www.google.ca/?gws_rd=ssl#q=android+java+reader+app

Try any of those ones and hopefully one could work. Alternatively, you can read examples provided in Github from different libraries, examples provided in the Processing's webpage or in the Processing's reference webpage.

Kf

Not sure what you mean. Maybe you are not aware that iOS users are able to use P5.js in several ways. 1. With the P5.js app available on the App Store 2. Within smart BASIC, a comprehensive and powerful language that runs on all iOS devices.

I was hoping I would hear back from a moderator of the Forum.

I was wondering after I code my program what the steps are to get an app published on the google play store. Keyword: steps

there is a live wallpapers tutorial here:

http://android.processing.org/tutorials/wallpapers/index.html

but the fact that the wallpapers choice is a menu option and daydreaming isn't available makes me think that you can't do this (yet).

there was, i think, a 'daydream using live wallpaper' app on the app store.

I have used roboVM and lib gdx to make games (not with the processing lib). So i though it would be worth trying processing and roboVM.

I imported processing into eclipse, made a simple sketch..it ran fine!

Then I tried to build the sketch using roboVM on my IPhone 6. I managed to get the app running but the graphic where not displaying. I plan to take another look into this soon and will update when i have news. I would be great for the processing community to venture into the app store!

I was just wondering if it would be possible to create an app with p5.js, wrap it with phonegap and then submit it to the app store ? From what I gathered from my searches, it could possible, but I can't find any example or details.

So... Can it be done? Did anyone succeed?

I've been using Processing for quite a while now but I'm beginning to learn p5.js and javascript, so sorry if my question is naive / doesn't make sense. I'd like to create a very simple game (Pong simple) for iOS and, to be honest, I can't convince myself to learn Swift...

Alright, thanks! :)

After being very excited to make some actual tools that I can share with someone via a single file they "double-click" to execute I am being asked if others can buy these tools.

I would like to find out if anyone has any experience putting the exported apps made in Processing on the Apple App Store for the Mac?

I am thinking it would be pretty straightforward. They run the same as a bone fide app that anyone has downloaded. But are there any extra steps to getting it on there? Other than becoming a registered Apple Developer of course.

If there isn't a tutorial then I think there should be one!

hi~ I am using Sublime text with Processing. recently I find CodeRunner editor on app store. I thing that is very simple editor and use other programming language. so, I hope to use CodeRunner editor for Processing. but I don't know...how can i use that.. thank you~

https://coderunnerapp.com/ https://coderunnerapp.com/support#addlanguages

@ChuiGum: Uploading the app to the Amazon App Store should not make a difference for Google Play. As far as I am aware, both of these services allow apps to be dually hosted on each other.

I tested uploading an APK built and signed with APDE to a new (temporary) Google Play application and it produced no errors.

What error message do you receive? If you are updating an existing application, did you sign it with the same keystore as you used the first time? An upload failure should have nothing to do with Processing and/or APDE. Most likely, you did something wrong with the manifest file or with signing.

Can I upload the same .apk as I did to Google Play to the Amazon App Store? It was failing, so I was wondering if it was Processing itself or something else?

You need a tester? :) I would love to try that out. Now we finally have a way to get porn in the app store :)